test: run the --bail JUnit regression tests concurrently and assert the full report - #41423

Problem

• test/regression/issue/26851.test.ts takes up to 12s on the debian 13 x64-asan lane. It is in the slowest 5 percent of test files. Both tests spawn a bun test --bail child, and they ran one after the other.
• On that lane each child exits 134, not 1. bun test --bail exits with a bare exit(1), so the BUN_DESTRUCT_VM_ON_EXIT teardown is skipped and LeakSanitizer aborts the child after several seconds of report symbolization (bun test exits after failures are not LSan-clean (bail and failing runs abort under detect_leaks=1) #32183, fix open in bun test: exit through the VM teardown path when --bail stops a run #39010). Locally each child takes 0.3s. Under the lane environment it takes 3.8s. The old expect(exitCode).not.toBe(0) hid the abort.
• The old assertions only checked that the JUnit file exists and contains four substrings. They did not check the counters, the failure element, the bail message, or the exit code.

Fix

• Add the file to test/no-validate-leaksan.txt, next to test/regression/issue/12250.test.ts, which is on the list for the same bail exit. The entry is to be removed when bun test: exit through the VM teardown path when --bail stops a run #39010 lands. Without LeakSanitizer the children exit 1 in well under a second.
• Run both tests with test.concurrent. Each test owns its tempDir and its outfile, so the two children overlap. A shared runBailWithJUnit helper does the spawn and reads the report.
• Assert the full JUnit document with toMatchInlineSnapshot: the testsuites root with tests, assertions, failures and skipped, one testsuite per file that ran, the testcase names and lines, and the failure element with its type and message. Only the time and hostname attributes are normalized. Assert the child output before the exit code: stdout is the version banner, stderr has the (pass)/(fail) lines, the Ran N tests across M files. summary and Bailed out after 1 failure. The exit code is exactly 1.
• Add a second test after the failing one, and a third test file c_never.test.ts. Neither appears in stderr or in the report. This proves that --bail stopped the run and that the report holds only what ran. Discovery order of the root directory is sorted by name, so a_pass runs before b_fail.
• Verified: bun bd test test/regression/issue/26851.test.ts. Plain, 3 runs: before 2.66s to 3.14s wall, after 2.27s to 2.44s wall. With the ASAN lane environment (BUN_DESTRUCT_VM_ON_EXIT=1, detect_leaks=1): before 9.53s with both children exiting 134, after the list entry that environment is not applied. Also passes with USE_SYSTEM_BUN=1.

Background

• --bail makes bun test stop after the first failing test. bun test should produce reporter outfile when --bail is specified #26851 was about the JUnit outfile not being written in that case.
• test/no-validate-leaksan.txt lists test files for which scripts/runner.node.mjs does not set BUN_DESTRUCT_VM_ON_EXIT and detect_leaks=1 on the ASAN lane.
• make more tests faster #33704 also adds test.concurrent to this file as part of a broad speed pass. This PR is scoped to the one file and also strengthens the assertions.

[auto-merge] gate passed · iteration 1 · 2 files touched

Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/regression/issue/26851.test.ts'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "test/regression/issue/26851.test.ts"
bun test v1.4.3 (e0a2b82fd)

test/regression/issue/26851.test.ts:
(pass) --bail writes JUnit reporter outfile [356.62ms]
(pass) --bail writes JUnit reporter outfile with multiple files [349.11ms]

 2 pass
 0 fail
 2 snapshots, 15 expect() calls
Ran 2 tests across 1 file. [2.42s]
Exit: 0

test/no-validate-leaksan.txt        |   3 +
 test/regression/issue/26851.test.ts | 140 +++++++++++++++++++++---------------
 2 files changed, 87 insertions(+), 56 deletions(-)

gate history · 3 passed · 0 rejected · iteration 1

file                                 reads  edits  tests
test/no-validate-leaksan.txt             1      1     25
test/regression/issue/26851.test.ts      3      5     19

root cause · written by the author bot

The regression test for issue #26851 was slow because each of its five tests spawned a separate bun test child one after another, and under ASAN every child paid several seconds of startup, while the assertions only checked that the JUnit outfile existed. The fix runs the independent cases concurrently through a shared runner that captures stdout, stderr, the exit code and the report once per fixture set, then asserts the full normalized JUnit document shape, the bail message, the summary line and the exact exit code. A follow-up strips the CI environment variables that make the reporter …